{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Scipy是世界上著名的Python开源科学计算库，建立在Numpy之上。  \n",
    "它增加的功能包括数值积分、最优化、统计和一些专用函数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本篇文档包含以下内容：  \n",
    "1、文件的输入/输出；  \n",
    "2、统计；  \n",
    "3、信号处理；  \n",
    "4、最优化；  \n",
    "5、插值。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## scipy.io文件的输入和输出"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1., 1., 1.],\n",
       "       [1., 1., 1.],\n",
       "       [1., 1., 1.]])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import scipy.io as spio\n",
    "import numpy as np\n",
    "\n",
    "a = np.ones((3,3))\n",
    "a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 保存文件\n",
    "spio.savemat('file.mat',{'a':a})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'__header__': b'MATLAB 5.0 MAT-file Platform: nt, Created on: Mon Jun 24 08:41:15 2019',\n",
       " '__version__': '1.0',\n",
       " '__globals__': [],\n",
       " 'a': array([[1., 1., 1.],\n",
       "        [1., 1., 1.],\n",
       "        [1., 1., 1.]])}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 导入文件\n",
    "data = spio.loadmat('file.mat',struct_as_record=False)\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 载入txt文件：numpy.loadtxt()/numpy.savetxt();  \n",
    "- 智能导入txt/csv文件：numpy.genfromtxt()/numpy.recfromcsv();  \n",
    "- 高速，有效率但Numpy特有的二进制格式：numpy.save()/numpy.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 统计"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 分析随机数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-0.34528157,  2.08815987,  1.82869524,  0.12579845,  0.29959837,\n",
       "        0.87003351,  0.61437405, -0.22592588, -0.05347454, -0.66447418,\n",
       "        0.26893793, -1.17133621, -0.99217852, -0.16566653, -0.31556221,\n",
       "       -0.61260345,  2.10420798, -0.08411061, -1.32425702, -1.35093201,\n",
       "       -0.57164146, -0.77179966,  0.89423545, -0.45024258,  1.96409248,\n",
       "        0.32634276,  0.5014092 , -0.78962137, -1.17028196, -0.6898958 ,\n",
       "       -0.20760081,  0.31242949,  0.34431254, -1.66208393, -0.69620469,\n",
       "       -0.80432335,  0.66546073,  0.43927958, -2.06716681, -1.68545625,\n",
       "       -0.62932207,  0.83063667, -0.63510766, -0.75105419, -0.37464606,\n",
       "        0.75066259, -0.39648871,  0.02486525, -0.03640586, -1.77585868,\n",
       "       -1.1846139 ,  0.63991387,  0.45557378,  0.29890528, -0.36515162,\n",
       "       -0.07602407, -1.11927194, -0.54624045, -1.93463211,  1.7988412 ,\n",
       "       -0.80051722, -1.2777623 , -0.3955567 , -0.28165796, -0.37553267,\n",
       "        0.62394455, -0.65192526,  1.53046636,  0.19556439,  0.19000304,\n",
       "        0.83298353, -0.85509088, -1.06213577,  0.02191224, -1.48563573,\n",
       "       -1.55421383,  0.13396109,  0.09385745, -0.99741964,  1.15849905,\n",
       "        1.52784658,  0.18415285,  0.06011277, -0.2497277 ,  0.27635447,\n",
       "        0.11870793, -0.28970615, -1.07188897,  0.7914313 , -0.82535729,\n",
       "       -1.20593861, -1.6947422 , -1.489494  , -0.52776726,  1.91910599,\n",
       "        0.27234179,  1.72716244,  0.12509805, -1.60355805,  0.35831877,\n",
       "        0.33543938,  0.97415293, -0.22540347, -0.23812768,  0.36397863,\n",
       "        1.16635474,  0.2633748 ,  0.29188299, -1.79450129, -0.52444865,\n",
       "        0.45543597, -0.50665568,  2.05284558, -0.18958524, -0.51314231,\n",
       "        0.20309266, -0.67378024, -0.23070302,  1.05336267, -0.68858301,\n",
       "       -0.28228495, -2.2602363 ,  0.92954764, -0.7611345 , -0.97234988,\n",
       "        0.53380783,  0.323729  ,  0.43729087,  0.62869055, -0.75345493,\n",
       "       -1.19687574,  0.40876588, -0.78169945,  0.72306749, -0.04835109,\n",
       "        0.92836107, -2.46785637,  1.426109  ,  0.42292198,  0.6790397 ,\n",
       "       -0.35038729,  0.44151571, -1.10101024,  0.87943611,  0.25112133,\n",
       "       -1.61618662, -0.18183988, -0.16636676,  0.56093922,  0.2043981 ,\n",
       "        0.22681448,  0.01960589, -0.29442441,  0.15536511, -0.82564369,\n",
       "        1.84175162,  0.02025636, -0.84492469,  0.07920314,  0.88917349,\n",
       "       -1.18388493,  0.58071187,  0.35009309, -0.33620038, -0.9535673 ,\n",
       "        0.97984537,  0.59215577,  0.28679291, -0.06083778, -2.30092839,\n",
       "       -0.88326564,  0.94451037, -0.41012544, -0.08216614, -0.00677103,\n",
       "        1.025261  ,  0.87807134,  1.47800312,  0.2001529 , -2.0944058 ,\n",
       "       -1.70449716,  0.18652813, -1.85339278,  0.45355188,  1.07332596,\n",
       "       -0.39733117, -0.04501592, -0.06074738, -0.18955462,  0.00847396,\n",
       "        1.63624608, -0.1622361 ,  0.96261397,  0.27628495,  1.04492643,\n",
       "        0.02354394, -1.33957287, -0.21245787, -0.67171308, -0.33991163,\n",
       "        0.99100065, -0.4080037 ,  0.19756771,  2.28727573, -1.43221552,\n",
       "       -0.21516851, -0.86874916,  0.30836539, -1.21659947, -1.58274134,\n",
       "       -0.04328858,  1.41708913,  0.18090387, -0.78080175, -0.71492051,\n",
       "       -0.19426371, -1.43239547, -0.06015262,  0.32034996,  0.85516801,\n",
       "        0.65053414, -0.16947186, -0.84887182, -1.99896941,  0.67290269,\n",
       "        0.39308709, -0.08339639, -2.22835113,  0.7853648 ,  0.38141612,\n",
       "       -0.2101491 ,  1.60494726, -1.16936626, -0.61422644,  0.00607913,\n",
       "       -0.99347397, -0.88032949,  0.11826677, -1.96246394,  0.66865396,\n",
       "        0.33205664, -0.39617916, -1.03367975,  0.45113843,  1.21898769,\n",
       "        0.16649427, -1.99829489,  1.23355022,  0.76805953,  0.27959036,\n",
       "        0.30959845, -0.65933125,  1.04665333,  0.24710537, -0.80276685,\n",
       "       -1.47215926, -0.10430411, -0.89691011, -0.43188261,  0.54588152,\n",
       "       -0.69030325,  1.87391232,  0.7324781 , -0.52838597, -0.23586986,\n",
       "        1.55693826, -0.79454582,  0.08054554,  1.06295851, -0.57215243,\n",
       "       -0.25224927, -1.9299535 ,  0.51631405,  0.38542765, -2.33111576,\n",
       "       -0.78705994,  1.04950191,  0.18240965, -0.98133385,  0.12102649,\n",
       "        1.71413552, -1.1163973 , -0.80540615,  1.87521635, -0.29505187,\n",
       "       -0.25115647, -0.8597376 ,  0.4862349 , -0.5314083 ,  0.12370722,\n",
       "       -0.78225292, -1.63426277,  0.87116984,  0.51191804, -1.60431815,\n",
       "       -0.48113339,  0.60972783, -1.03795294, -0.28014336,  0.17965333,\n",
       "        0.68186732, -0.23651859, -0.79181201,  0.31102847, -2.09683586,\n",
       "       -1.84244033,  2.13669164,  1.64161679, -1.27472603,  1.22136456,\n",
       "        0.49499692, -0.70803928, -0.68807544, -1.05610184, -0.15623005,\n",
       "       -0.32195552,  0.31713102, -1.93718086,  1.13872194,  0.86546858,\n",
       "       -1.94516103,  0.24704356,  0.70653051, -0.36908773, -0.57581415,\n",
       "        0.94862333,  0.66676554, -0.04644496, -0.35615446, -0.20821175,\n",
       "        1.2079406 ,  1.58047661, -0.47361416, -0.29266155, -1.58019507,\n",
       "        1.3316992 ,  0.49244031,  1.31840715, -0.73421761,  1.24757777,\n",
       "        0.78241806,  0.62027034,  0.41493859, -1.09480987,  0.28241494,\n",
       "        0.48842375,  1.00409894,  0.42600207,  0.30754962, -2.13126037,\n",
       "        0.91511558,  0.98430646, -2.41900097,  1.38431484,  0.70959103,\n",
       "       -0.84787956, -1.6061129 ,  0.89299501,  0.55347395,  0.65537306,\n",
       "        1.51103902,  1.99106415,  1.12003962, -1.08643696, -1.05236532,\n",
       "       -0.11969793,  0.26394532,  1.22858845, -1.23196127, -0.18514828,\n",
       "       -1.26606079, -1.40414391,  0.30458866, -0.19368087, -0.30382459,\n",
       "       -0.95829672, -0.87782655,  1.21554043, -0.58159907, -0.12693706,\n",
       "        0.41867647,  0.96214029,  1.24208095,  0.58559694,  2.40208158,\n",
       "       -0.31697621,  1.60944704,  0.18410533,  0.45701902,  0.08648072,\n",
       "       -1.39994042,  0.35754595,  1.23934705,  2.17904094,  0.80199327,\n",
       "        0.33352428, -1.12724347,  0.54278549, -1.11909606, -0.99175426,\n",
       "       -0.39550701, -0.59680819, -0.1601334 ,  1.55172777, -0.23359744,\n",
       "       -0.81618234, -1.4932824 , -0.38238446, -0.43870063, -1.89691679,\n",
       "       -0.01727554, -0.30809047,  1.21728278, -0.52973468, -1.51366332,\n",
       "       -1.61047112, -1.46914598, -0.40886721, -0.09315859,  1.18702907,\n",
       "        1.12704444, -1.28494676, -1.41823526,  0.38151442,  0.45379587,\n",
       "       -1.22549963,  1.05007197, -0.26809535,  1.86038223,  1.55695522,\n",
       "       -1.40962187, -0.01553562, -0.99443984,  1.08769777, -0.39485825,\n",
       "        0.18385408, -0.30143384, -0.04686673,  0.14474325,  0.19255177,\n",
       "       -1.15800782, -0.49721805, -0.77811205,  0.85494863, -0.80977269,\n",
       "       -0.30041946,  0.11644208,  0.04911225,  1.08275043, -0.65855558,\n",
       "       -0.49154538,  1.59997911, -0.33621286, -0.33206155,  0.36696096,\n",
       "        1.64634715, -0.11330233,  1.52026137, -0.59534552, -0.41141016,\n",
       "        0.27270449, -0.61662218,  2.562799  ,  0.3150998 ,  0.98051175,\n",
       "        0.82760819,  0.26374172, -0.73274947, -0.38078076,  1.37381386,\n",
       "       -0.34655612,  0.2873232 , -0.32150825, -0.92427114, -0.50066042,\n",
       "       -0.05265025, -0.73355302,  0.16671722, -0.04602939,  1.54230973,\n",
       "        1.19224774, -0.35931427, -1.38703101, -0.45746818,  0.39847129,\n",
       "       -0.79738991,  0.02645559, -0.64839847,  0.18657374,  0.65209423,\n",
       "       -0.91113111,  0.58899239,  0.2144381 , -0.01279646,  0.03406188,\n",
       "        0.41519807,  0.06612646, -1.20887298,  0.12565491, -0.15706841,\n",
       "        0.70791082,  0.4310594 , -0.47287049,  0.04327748, -1.65089804,\n",
       "        1.5519427 , -0.43545358, -0.00635588, -1.41448457,  2.88146116,\n",
       "        1.43895527,  0.05241671,  0.64785818,  0.85687407, -0.03814247,\n",
       "       -0.76656295, -0.37442331, -1.01653633, -0.5704561 , -0.2566547 ,\n",
       "       -0.35777059,  0.1048866 ,  1.30841159, -1.3358326 , -0.64631197,\n",
       "       -0.10662386, -0.70772196,  0.30546175, -0.92544399,  1.37240847,\n",
       "       -0.12699716,  1.16538497, -0.5813093 , -0.92110314,  0.36145404,\n",
       "        0.17733288, -1.10752982,  0.01997102,  1.91308913, -1.5290556 ,\n",
       "        0.44868026,  0.00527793, -0.66915456,  0.17611573, -0.48381488,\n",
       "        0.00786151,  0.95669231,  0.37513694, -0.56742732, -0.29156508,\n",
       "       -0.77171233,  0.2693153 , -0.21483273, -0.24520194,  0.86606302,\n",
       "       -2.13022714,  0.89746988,  0.6786931 ,  0.68410249, -0.84595773,\n",
       "       -0.60226903, -0.82481677,  0.47458766,  0.4747039 ,  1.29629348,\n",
       "        0.2640413 , -1.84844224, -0.64230107,  1.78346779,  1.71108013,\n",
       "       -0.01527558,  0.68254291,  0.93065924, -0.97934383,  0.11841763,\n",
       "       -1.24817173,  0.63057099,  0.93157148, -0.95605653,  0.16343603,\n",
       "        0.96958366,  0.00480336, -0.44596102,  0.73072692,  0.87940161,\n",
       "       -0.61597746,  0.46788574,  0.90903262, -0.36502692,  1.62269255,\n",
       "        0.64726055, -1.85453045, -1.61455657, -0.11649329,  0.30563497,\n",
       "        0.82371214,  0.53472873,  0.45920627, -0.3921953 ,  0.19025013,\n",
       "        0.82244095,  0.88749923,  0.34916735,  1.21964804, -0.45311867,\n",
       "       -0.55810923,  2.21083753,  0.6368813 , -0.93028622,  1.85462009,\n",
       "        0.45822329,  1.44537315, -1.39139559,  2.02733815, -0.44157747,\n",
       "       -0.27188562,  0.50340948, -1.0042714 ,  1.96535039, -1.89310322,\n",
       "       -0.59497634,  0.08959413,  0.50545059,  0.9756999 ,  0.27857848,\n",
       "       -2.19476475, -0.49771205,  0.28494738, -0.64596017, -0.96846026,\n",
       "       -0.81953766, -0.27976642, -1.20464963,  1.1041236 ,  0.58698613,\n",
       "        0.1636564 ,  0.76245837, -0.2483711 , -0.96908513, -0.18801739,\n",
       "        2.11497019,  1.14508995, -0.45189107,  0.29263573,  0.35718269,\n",
       "       -0.30111048, -0.29162027, -1.26659791, -0.15537657, -0.21802139,\n",
       "       -0.29564373,  0.94560328, -0.40617154, -1.78953569, -0.36184243,\n",
       "       -0.07535188, -0.08289876, -0.67356937, -0.23200693, -1.20215636,\n",
       "        0.72536866, -0.14018736,  1.06193114, -0.48985851, -0.47050947,\n",
       "        0.41548683, -0.830564  , -2.37976397, -1.36379171, -0.88006414,\n",
       "        1.27128386, -0.35897533, -0.82385838,  0.21348168, -1.45047451,\n",
       "        1.083719  ,  1.03141456, -0.48222825, -0.93564771, -0.88037173,\n",
       "       -1.17136171,  0.63218486,  0.10170357,  0.28017608, -1.45069595,\n",
       "        0.21368893,  0.10466728, -0.39807254,  0.34272778, -1.58539967,\n",
       "        0.57599985, -1.43094754,  1.55027748, -0.18508467, -0.75005998,\n",
       "       -0.79081538, -0.88164305,  0.27410909, -0.77941806, -0.04359558,\n",
       "       -1.02903565,  0.16866236, -0.70111541,  1.25952545, -0.21362047,\n",
       "       -0.89260234,  0.32401714,  0.91140061, -0.16171641, -1.00208511,\n",
       "        0.01563178,  2.6188296 ,  0.28625719,  0.31480023, -0.59134255,\n",
       "       -0.45897283,  1.0539497 ,  1.76850355, -0.94463705,  1.25614992,\n",
       "        2.02590984, -0.8609187 ,  2.77789487, -1.24562074,  0.09925315,\n",
       "        0.20571053,  1.2649108 , -0.95692148,  1.56676771, -2.66266433,\n",
       "       -0.00405412, -1.1260683 ,  0.47936507,  0.76450562,  0.42314008,\n",
       "        2.54063722,  1.07960917,  0.64936087, -1.91419221,  0.21195796,\n",
       "        0.09214212,  0.1746923 ,  1.34973886,  1.98100017, -0.54081394,\n",
       "        2.08894221, -0.08472885,  0.9615003 ,  0.32242808, -0.63895309,\n",
       "       -0.16913533, -0.44805163,  0.87133634,  1.23922216,  0.75334413,\n",
       "       -1.30695948,  0.25137267, -0.5434644 , -0.83386154,  0.15083077,\n",
       "        0.20644013,  0.78243   , -1.30701892, -0.92003417, -1.57046377,\n",
       "        0.45937552,  1.60439267, -0.46164738, -0.9177952 ,  0.73462895,\n",
       "       -0.24658268,  1.47174485, -0.80322828, -0.28037512,  1.08957259,\n",
       "       -1.33072548,  1.1177233 , -2.0317448 , -1.09403017,  0.88015281,\n",
       "        0.74886738,  0.51324509,  0.78517376,  0.23683136, -2.14166181,\n",
       "       -1.74971673, -1.89057224,  0.3864179 ,  1.66930331, -0.96121467,\n",
       "       -0.44241556,  0.39887512, -1.78838665,  1.39927174,  0.09304574,\n",
       "       -0.15932639, -0.41989915,  0.02544559,  0.1757497 , -0.73122639,\n",
       "        1.17546583,  1.92680436,  0.18270913, -3.10049203,  0.51389824,\n",
       "        1.91786747,  1.05389567, -0.99079496,  0.34996254, -2.52476475,\n",
       "       -1.74720461, -1.41463693, -1.01965819,  1.50365298, -1.50880789,\n",
       "        1.27828272, -0.83571007, -0.68819916,  1.10894075, -1.53769421,\n",
       "        1.63364234,  1.06408151, -0.02518378,  0.69931238,  0.27072351,\n",
       "       -1.03303022,  0.4998533 ,  1.06534304, -0.80380836, -0.0191823 ,\n",
       "       -0.36190389,  1.14218815,  0.12230745,  0.42420851,  0.01624287,\n",
       "       -1.66767963, -1.30735178,  0.16251495,  0.04696505,  0.54297712,\n",
       "       -0.6265375 ,  0.39699728, -0.55525063, -0.77997401, -0.15617358,\n",
       "       -0.07456111,  0.56566069,  0.96315643, -0.56776021, -0.17756982,\n",
       "       -0.07056279, -0.49867163, -2.18978442, -0.835868  ,  0.42998044,\n",
       "        2.24464126, -0.26027234, -0.16537194,  0.45721978, -2.32351496,\n",
       "       -0.90185046,  1.439061  ,  0.11350054,  0.86391961, -0.89982707,\n",
       "       -0.11718306, -0.57653324, -0.07644854,  0.59263666,  0.60095397,\n",
       "        0.16109671, -0.43647524, -1.31641455, -0.65319914,  0.9048198 ,\n",
       "        0.38047493, -1.17344146,  1.88970774, -0.6744916 , -1.14214621,\n",
       "        0.82750096, -0.88587602, -0.84273629,  0.18748726,  1.60700741,\n",
       "       -1.28430447,  1.52111605, -2.18023175, -1.33935089, -1.52278615,\n",
       "        1.49309651,  1.21891488, -0.55982484,  0.74816588, -1.17839271,\n",
       "        0.44795985,  2.45037254, -0.98373965,  0.64005916, -1.09375564])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import scipy.stats as stats\n",
    "\n",
    "# 生成正态分析的随机数\n",
    "generated = stats.norm.rvs(size=900)\n",
    "generated"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 用正态分布去拟合生成的数据，得到均值和标准差\n",
    "Mean,std = stats.norm.fit(generated)\n",
    "# print('Mean=',Mean,'\\nstd=',std)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-0.03791315740642637"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Mean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9943876756715423"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "std"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 偏度"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "偏度（skewnes）描述的是概率分布的偏斜程度，我们需要做一个偏度检验。该检验有两个返回值，其中第二个返回值是p-value, 即观察到的数据服从正态分布的概率，取值为0-1。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SkewtestResult(statistic=0.5209439390256865, pvalue=0.6024058252666132)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stats.skewtest(generated)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "即我们有`pvalue`的把握认为其服从正态分布。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 峰度\n",
    "峰度（kurtosis）描述的是概率分布的陡峭程度。该检验和偏度检验类似。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KurtosistestResult(statistic=-0.9869827063753087, pvalue=0.32365111656437995)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stats.kurtosistest(generated)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 正态性检验\n",
    "正态性检验（normality test）可以检验数据服从正态分布的程度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "NormaltestResult(statistic=1.2455174502915272, pvalue=0.5364624418969589)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stats.normaltest(generated)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 得到数据所在区域中某一百分比处的数值\n",
    "利用Scipy我们可以很方便地得到数据所在区域中某一百分比处的数值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.6232400406406529"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 得到95%处的数值score at percentile\n",
    "stats.scoreatpercentile(generated,95)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "85.11111111111111"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 同样，可以反过来得到数值所在的百分比：percentile of score\n",
    "stats.percentileofscore(generated,1)\n",
    "# stats.percentileofscore(generated,99999)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 使用matplot绘制生成数据的分布直方图"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADNFJREFUeJzt3X+onYV9x/H3p+q6YTtUcpVM424ZYdSNzZYgQsdw2LVWR2P/cChjDZ2QFZRZ2KBZC3M/EFLGutGxyTKURrB2ghUDcZsu63D9w9Yozmqja+hSTRNMWtdWETai3/1xn9A7e8299/zIuffr+wWXc85zn3ue72P07ZPnnPPcVBWSpL7eNusBJEnTZeglqTlDL0nNGXpJas7QS1Jzhl6SmjP0ktScoZek5gy9JDV35qwHANiwYUPNz8/PegxJWlcef/zx71bV3HLrrYnQz8/Ps3///lmPIUnrSpJvr2Q9T91IUnOGXpKaM/SS1Jyhl6TmDL0kNWfoJak5Qy9JzRl6SWrO0EtSc2vik7HSWja/Y+9Mtnto5zUz2a768Yhekpoz9JLUnKGXpOYMvSQ1Z+glqTlDL0nNGXpJas7QS1Jzhl6SmjP0ktScoZek5gy9JDVn6CWpOUMvSc0ZeklqbtnQJ9mU5MtJDiR5Jsktw/Lzkjyc5JvD7bnD8iT5XJKDSZ5K8t5p74Qk6c2t5Ij+BPD7VfVu4HLgpiSXADuAfVW1Gdg3PAb4ELB5+NoO3D7xqSVJK7Zs6KvqaFU9Mdx/GTgAXAhsBXYPq+0Grh3ubwXuqgWPAuck2TjxySVJK7Kqc/RJ5oH3AF8FLqiqo7DwPwPg/GG1C4EXFv3Y4WHZG59re5L9SfYfP3589ZNLklZkxaFP8g7gPuATVfXDU626xLL6sQVVu6pqS1VtmZubW+kYkqRVWlHok5zFQuTvrqovDYtfPHlKZrg9Niw/DGxa9OMXAUcmM64kabVW8q6bAHcAB6rqs4u+tQfYNtzfBjywaPlHh3ffXA784OQpHknS6XfmCtZ5H/DbwNeTPDks+xSwE7g3yY3A88B1w/ceBK4GDgKvAh+b6MSSpFVZNvRV9RWWPu8OcOUS6xdw05hzSZImxE/GSlJzKzl1I83c/I69sx5BWrc8opek5gy9JDVn6CWpOUMvSc0ZeklqztBLUnOGXpKaM/SS1Jyhl6TmDL0kNWfoJak5Qy9JzRl6SWrO0EtSc4Zekpoz9JLUnKGXpOYMvSQ1Z+glqTlDL0nNGXpJas7QS1Jzhl6SmjP0ktScoZek5gy9JDVn6CWpOUMvSc0ZeklqztBLUnOGXpKaM/SS1Jyhl6TmDL0kNWfoJak5Qy9JzS0b+iR3JjmW5OlFy/44yXeSPDl8Xb3oe3+Y5GCS55J8cFqDS5JWZiVH9J8Hrlpi+V9W1aXD14MASS4Brgd+YfiZv01yxqSGlSSt3rKhr6pHgJdW+HxbgS9W1f9U1X8BB4HLxphPkjSmcc7R35zkqeHUzrnDsguBFxatc3hY9mOSbE+yP8n+48ePjzGGJOlURg397cDPAZcCR4G/GJZniXVrqSeoql1VtaWqtszNzY04hiRpOSOFvqperKrXqup14O/50emZw8CmRateBBwZb0RJ0jhGCn2SjYsefgQ4+Y6cPcD1Sd6e5F3AZuBr440oSRrHmcutkOQe4ApgQ5LDwK3AFUkuZeG0zCHgdwGq6pkk9wLfAE4AN1XVa9MZXZK0EsuGvqpuWGLxHadY/zbgtnGGkiRNjp+MlaTmDL0kNWfoJak5Qy9JzRl6SWpu2XfdSIvN79g76xHeMmb1z/rQzmtmsl1Nj0f0ktScoZek5gy9JDVn6CWpOUMvSc0ZeklqztBLUnOGXpKaM/SS1Jyhl6TmDL0kNWfoJak5Qy9JzRl6SWrO0EtSc4Zekpoz9JLUnKGXpOYMvSQ1Z+glqTlDL0nNGXpJas7QS1Jzhl6SmjP0ktScoZek5gy9JDVn6CWpOUMvSc0ZeklqztBLUnPLhj7JnUmOJXl60bLzkjyc5JvD7bnD8iT5XJKDSZ5K8t5pDi9JWt5Kjug/D1z1hmU7gH1VtRnYNzwG+BCwefjaDtw+mTElSaNaNvRV9Qjw0hsWbwV2D/d3A9cuWn5XLXgUOCfJxkkNK0lavVHP0V9QVUcBhtvzh+UXAi8sWu/wsEySNCOTfjE2SyyrJVdMtifZn2T/8ePHJzyGJOmkUUP/4slTMsPtsWH5YWDTovUuAo4s9QRVtauqtlTVlrm5uRHHkCQtZ9TQ7wG2Dfe3AQ8sWv7R4d03lwM/OHmKR5I0G2cut0KSe4ArgA1JDgO3AjuBe5PcCDwPXDes/iBwNXAQeBX42BRmliStwrKhr6ob3uRbVy6xbgE3jTuUJGly/GSsJDVn6CWpOUMvSc0ZeklqztBLUnOGXpKaM/SS1Jyhl6TmDL0kNWfoJak5Qy9JzRl6SWrO0EtSc4Zekpoz9JLUnKGXpOYMvSQ1Z+glqbllf5WgpLeW+R17Z7btQzuvmdm2O/OIXpKaM/SS1Jyhl6TmDL0kNWfoJak5Qy9JzRl6SWrO0EtSc4Zekprzk7Hr0Cw/uShp/fGIXpKaM/SS1Jyhl6TmDL0kNWfoJak5Qy9JzRl6SWrO0EtSc4Zekpob65OxSQ4BLwOvASeqakuS84B/AOaBQ8BvVtV/jzemJGlUkzii/7WqurSqtgyPdwD7qmozsG94LEmakWmcutkK7B7u7wauncI2JEkrNG7oC3goyeNJtg/LLqiqowDD7fljbkOSNIZxr175vqo6kuR84OEkz670B4f/MWwHuPjii8ccQ5L0ZsY6oq+qI8PtMeB+4DLgxSQbAYbbY2/ys7uqaktVbZmbmxtnDEnSKYwc+iRnJ3nnyfvAB4CngT3AtmG1bcAD4w4pSRrdOKduLgDuT3Lyeb5QVf+U5DHg3iQ3As8D140/piRpVCOHvqq+BfzyEsu/B1w5zlCSpMnxk7GS1Jyhl6TmDL0kNWfoJak5Qy9JzRl6SWrO0EtSc4Zekpoz9JLUnKGXpOYMvSQ1Z+glqblxf/GIJE3M/I69M9nuoZ3XzGS7p4tH9JLUnKGXpOYMvSQ1Z+glqTlfjB3DrF44kqTV8Ihekpoz9JLUnKGXpOYMvSQ1Z+glqTlDL0nNGXpJas7QS1Jzhl6SmjP0ktScoZek5gy9JDVn6CWpOUMvSc0ZeklqztBLUnP+4hFJb3mz/CVCh3ZeM/VteEQvSc0ZeklqztBLUnNTC32Sq5I8l+Rgkh3T2o4k6dSm8mJskjOAvwF+HTgMPJZkT1V9Y9LbmuWLKJK0HkzriP4y4GBVfauq/hf4IrB1StuSJJ3CtEJ/IfDCoseHh2WSpNNsWu+jzxLL6v+tkGwHtg8PX0ny3IRn2AB8d8LPOQtd9gPcl7XKfZmhfOZNv7WSffnZlWxjWqE/DGxa9Pgi4MjiFapqF7BrStsnyf6q2jKt5z9duuwHuC9rlfuyNk1yX6Z16uYxYHOSdyX5CeB6YM+UtiVJOoWpHNFX1YkkNwP/DJwB3FlVz0xjW5KkU5vatW6q6kHgwWk9/wpM7bTQadZlP8B9Wavcl7VpYvuSqlp+LUnSuuUlECSpubahT/JnSZ5K8mSSh5L8zKxnGlWSP0/y7LA/9yc5Z9YzjSrJdUmeSfJ6knX57ogul/dIcmeSY0menvUs40iyKcmXkxwY/t26ZdYzjSrJTyb5WpL/GPblTybyvF1P3ST56ar64XD/94BLqurjMx5rJEk+APzr8CL3ZwCq6pMzHmskSd4NvA78HfAHVbV/xiOtynB5j/9k0eU9gBumcXmPaUvyq8ArwF1V9YuznmdUSTYCG6vqiSTvBB4Hrl2nfyYBzq6qV5KcBXwFuKWqHh3nedse0Z+M/OBs3vCBrfWkqh6qqhPDw0dZ+FzCulRVB6pq0h+OO53aXN6jqh4BXpr1HOOqqqNV9cRw/2XgAOv0k/i14JXh4VnD19jtaht6gCS3JXkB+C3gj2Y9z4T8DvCPsx7iLczLe6xhSeaB9wBfne0ko0tyRpIngWPAw1U19r6s69An+ZckTy/xtRWgqj5dVZuAu4GbZzvtqS23L8M6nwZOsLA/a9ZK9mUdW/byHpqNJO8A7gM+8Ya/0a8rVfVaVV3Kwt/cL0sy9mm1df07Y6vq/Stc9QvAXuDWKY4zluX2Jck24DeAK2uNv7Cyij+X9WjZy3vo9BvOZ98H3F1VX5r1PJNQVd9P8m/AVcBYL5iv6yP6U0myedHDDwPPzmqWcSW5Cvgk8OGqenXW87zFeXmPNWZ4AfMO4EBVfXbW84wjydzJd9Ul+Sng/UygXZ3fdXMf8PMsvMPj28DHq+o7s51qNEkOAm8HvjcsenQdv4PoI8BfA3PA94Enq+qDs51qdZJcDfwVP7q8x20zHmkkSe4BrmDhKokvArdW1R0zHWoESX4F+Hfg6yz89w7wqeHT+etKkl8CdrPw79bbgHur6k/Hft6uoZckLWh76kaStMDQS1Jzhl6SmjP0ktScoZek5gy9JDVn6CWpOUMvSc39H/y+GHgVJ8EGAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import scipy.stats as stats \n",
    "\n",
    "plt.hist(generated)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 样本对比（比较股票对数收益率）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'get_price' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-10-9fc2bc117277>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[0;32m      2\u001b[0m \u001b[1;32mimport\u001b[0m \u001b[0mscipy\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstats\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0mstats\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      3\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 4\u001b[1;33m \u001b[0mprice\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mget_price\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'000001.XSHE'\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;34m'601398.XSHE'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mstart_date\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'2016-01-01'\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mend_date\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'2017-01-01'\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mfields\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'close'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      5\u001b[0m \u001b[0mprice_000001\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdiff\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlog\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0marray\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mprice\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'000001.xshe'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      6\u001b[0m \u001b[0mprice_601398\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdiff\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlog\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0marray\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mprice\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'601398.xshe'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mNameError\u001b[0m: name 'get_price' is not defined"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import scipy.stats as stats\n",
    "\n",
    "price = get_price(['000001.XSHE','601398.XSHE'],start_date='2016-01-01',end_date='2017-01-01',fields='close')\n",
    "price_000001 = np.diff(np.log(np.array(price['000001.xshe'])))\n",
    "price_601398 = np.diff(np.log(np.array(price['601398.xshe'])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 均值检验\n",
    "**均值检验**可以检验两组不同的样本是否有相同的均值。返回值有两个，其中第2个为p-value，取值范围为0-1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'price_000001' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-11-671164edc388>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mstats\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mttest_ind\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mprice_000001\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mprice_601398\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m: name 'price_000001' is not defined"
     ]
    }
   ],
   "source": [
    "stats.ttest_ind(price_000001,price_601398)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Kolmogorov-Smirnov检验\n",
    "Kolmogorov-Smirnov检验可以判断两组样本同分布的可能性。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'price_000001' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-12-f1089f5c8032>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mstats\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mks_2samp\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mprice_000001\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mprice_601398\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m: name 'price_000001' is not defined"
     ]
    }
   ],
   "source": [
    "stats.ks_2samp(price_000001,price_601398)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Jarque-Bera正态性检验\n",
    "在两支股票对数收益率的差值上运用Jarque-Bera正态性检验。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'price_000001' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-13-0f7c631b7380>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mstats\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mjarque_bera\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mprice_000001\u001b[0m \u001b[1;33m-\u001b[0m \u001b[0mprice_601398\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;33m-\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m: name 'price_000001' is not defined"
     ]
    }
   ],
   "source": [
    "stats.jarque_bera(price_000001 - price_601398)[-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 信号处理\n",
    "### 检验股价的线性趋势"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'get_price' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-15-f4b955f5b3be>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[0;32m      6\u001b[0m \u001b[1;32mfrom\u001b[0m \u001b[0mmatplotlib\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdates\u001b[0m \u001b[1;32mimport\u001b[0m \u001b[0mMonthLocator\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      7\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 8\u001b[1;33m \u001b[0mprice\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mget_price\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'000001.XSHE'\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mstart_date\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'2016-01-01'\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mend_date\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'2017-01-01'\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mfields\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'close'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m: name 'get_price' is not defined"
     ]
    }
   ],
   "source": [
    "from datetime import date,datetime,time\n",
    "from scipy import signal\n",
    "import pandas as pd\n",
    "from matplotlib.dates import DateFormatter\n",
    "from matplotlib.dates import DayLocator\n",
    "from matplotlib.dates import MonthLocator\n",
    "\n",
    "price = get_price('000001.XSHE',start_date='2016-01-01',end_date='2017-01-01',fields='close')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'price' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-17-bd25dcc25ffa>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0my\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0msignal\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdetrend\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mprice\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      2\u001b[0m \u001b[0mtrend\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mSeries\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0marray\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mprice\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;33m-\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mindex\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mprice\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mindex\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      3\u001b[0m \u001b[0mtrend\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mNameError\u001b[0m: name 'price' is not defined"
     ]
    }
   ],
   "source": [
    "y = signal.detrend(price)\n",
    "trend = pd.Series(np.array(price) - y, index = price.index)\n",
    "trend"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function detrend in module scipy.signal.signaltools:\n",
      "\n",
      "detrend(data, axis=-1, type='linear', bp=0)\n",
      "    Remove linear trend along axis from data.\n",
      "    \n",
      "    Parameters\n",
      "    ----------\n",
      "    data : array_like\n",
      "        The input data.\n",
      "    axis : int, optional\n",
      "        The axis along which to detrend the data. By default this is the\n",
      "        last axis (-1).\n",
      "    type : {'linear', 'constant'}, optional\n",
      "        The type of detrending. If ``type == 'linear'`` (default),\n",
      "        the result of a linear least-squares fit to `data` is subtracted\n",
      "        from `data`.\n",
      "        If ``type == 'constant'``, only the mean of `data` is subtracted.\n",
      "    bp : array_like of ints, optional\n",
      "        A sequence of break points. If given, an individual linear fit is\n",
      "        performed for each part of `data` between two break points.\n",
      "        Break points are specified as indices into `data`.\n",
      "    \n",
      "    Returns\n",
      "    -------\n",
      "    ret : ndarray\n",
      "        The detrended input data.\n",
      "    \n",
      "    Examples\n",
      "    --------\n",
      "    >>> from scipy import signal\n",
      "    >>> randgen = np.random.RandomState(9)\n",
      "    >>> npoints = 1000\n",
      "    >>> noise = randgen.randn(npoints)\n",
      "    >>> x = 3 + 2*np.linspace(0, 1, npoints) + noise\n",
      "    >>> (signal.detrend(x) - noise).max() < 0.01\n",
      "    True\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 可以通过以下代码查看去除趋势的作用\n",
    "help(signal.detrend)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'trend' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-19-c842f0b73259>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mtrend\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mplot\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m: name 'trend' is not defined"
     ]
    }
   ],
   "source": [
    "trend.plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 傅里叶分析\n",
    "对去除趋势后的信号进行滤波。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'y' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-21-32ed18cfdefc>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[0;32m      2\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      3\u001b[0m \u001b[1;31m# 运用傅里叶变换，得到信号的频道\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 4\u001b[1;33m \u001b[0mamps\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mabs\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfftpack\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfftshift\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfftpack\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mrfft\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0my\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m: name 'y' is not defined"
     ]
    }
   ],
   "source": [
    "from scipy import fftpack\n",
    "\n",
    "# 运用傅里叶变换，得到信号的频道\n",
    "amps = np.abs(fftpack.fftshift(fftpack.rfft(y)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 过滤噪音，如果某一频率分量的大小低于最强分贝10%，则过滤\n",
    "amp[amp<0.1*amps.max()] = 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 将过滤后的信号返回时域，并和去除趋势后的信号一起绘制出来\n",
    "plt.plot(price.index,y,label = 'detrended')\n",
    "plt.plot(price.index,-fftpack.irfft(fftpack.ifftshift(amps)),label='filtrend')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数学优化\n",
    "优化算法尝试寻求某一问题的最优解。例如找到函数的最大值或最小值，函数可以是线性的也可以是非线性的。解可能也有一定的约束，例如大于1。  \n",
    "在scipy.optimize模块中提供了一些优化算法，包括最小二乘法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 拟合正弦波"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在上一章节，我们为去除趋势后的数据建立了一个简单的滤波器。我们可以回忆一下，一个正弦波由4个参数决定Asin(ωx+φ)+k。  \n",
    "- A 振幅，当物体作轨迹符合正弦曲线的直线往复运动时，其值为行程的1/2；\n",
    "- (ωx+φ) 相位，反应变量y所处的状态；\n",
    "- φ 初相，x=0时的相位；反应在坐标系上则为图像的左右移动；\n",
    "- k 偏距，反应在坐标系上则为图像的上移或下移；\n",
    "- ω 角速度，控制正弦周期（单位角度内震动的次数）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 定义正弦波模型\n",
    "def residuals(p,y,x):\n",
    "    A,k,theta,b = p\n",
    "    err = y-A*np.sin(2*np.pi*k*k+theta)+b\n",
    "    return err"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 将滤波后的信号变换为时域\n",
    "filtered = -fftpack.irfft(fftpack.ifftshift(amps))\n",
    "pd.Series(filtered,index=price.index).plot()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "N = len(filtered)\n",
    "f = np.linspace(-N/2,N/2,N)\n",
    "p0 = [filtered.max(),f[amps.argmax()]/(2*N),0,0]\n",
    "p0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 调用leastsp函数\n",
    "from scipy import optimize\n",
    "\n",
    "plsq = optimize.leastsq(residuals,p0,args=(filtered,f))\n",
    "p=plsq[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure()\n",
    "plt.plot(price.index,y,'o',label='detrended')\n",
    "plt.plot(price.index,filtered,label='filtered')\n",
    "plt.plot(price.index,p[0]*np.sin(2*np.pi*f*p[1]+p[2]+p[3],'^',label='fit'))\n",
    "plt.legend(prop={'size':'x-large'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 积分（以高斯积分为例）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
